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Abstract 

As  it  makes  the  transition  from  the  industrial  age  to  the  information  age,  the  Department 
of  Defense,  like  other  government  agencies  and  indeed  like  the  global  business 
community  as  a  whole,  is  moving  aggressively  to  leverage  and  capitalize  on  advances  in 
information  technologies.  The  result  is  a  clear  trend  away  from  stand-alone  component 
systems  to  ones  that  are  richly  interconnected  and  increasingly  interdependent.  We  call 
these  “mega-systems.”  This  paper  focuses  on  the  engineering  of  this  class  of  systems 
which  is  characterized  by  increasing  scale,  the  nature  and  pace  of  change  of  the 
technologies  involved,  the  complexity  of  system  interactions  and,  perhaps  most 
important,  the  fact  that  a  single  organization  rarely  owns  and  has  complete  control  over 
the  mega-system.  We  hypothesize  that  engineering  these  mega-systems  is  inherently 
different  from  engineering  large-scale  but  essentially  well-bounded  monolithic  systems. 
We  develop  a  framework  to  understand  the  differences  between  well-bounded  systems 
and  this  new  class  of  systems.  On  the  basis  of  these  insights,  we  propose  an  approach  to 
engineering  mega-systems  that  emphasizes  experimentation  over  rigorous  requirements 
definition  and  continuous  evolution  over  a  “grand  design”. 

Introduction 

Demand  for  Agile,  Adaptive  Responses 

Several  factors  are  converging  to  fundamentally  change  the  nature  of  the  systems  that  are 
developed  and  fielded  to  the  U.S.  military  forces. 

The  strategic  environment  demands  agile  and  adaptive  response  to  a  wide  range  of  threats 
and  missions.  Responding  to  this  uncertainty  is  the  emerging  military  concept  of  network 
centric  warfare1  which  seeks  to  leverage  information  as  a  competitive  source  of  power. 
The  information  revolution,  on  which  this  concept  is  based,  provides  the  tools  by  which 
we  can  interconnect  a  wide  range  of  elements  and  provide  them  timely  information. 
Finally,  there  are  significant  changes  in  the  processes  by  which  the  Department  of 
Defense  (DOD)  intends  to  acquire  necessary  military  capability.2  These  converging 
trends  lead  to  a  growing  emphasis  on  large-scale,  richly  interconnected  capabilities  that 
bridge  traditional  organizational  and  functional  boundaries. 


'Alberts,  David  S.,  John  J.  Garstka,  and  Frederick  P.  Stein,  Network  Centric  Warfare,  2nd  Edition,  CCRP, 
1999. 

2  The  DOD  is  moving  from  a  bottom-up  requirements-based  process  to  a  top-down  capabilities-based 
process  and  is  implementing  the  Joint  Capabilities  Integration  and  Development  System  (JCIDS).  CJCSI 
3170. 01C  and  CJCSM  3170.01  can  be  found  at  http://www.dtic.mil/cics  directives. 
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Richly  networked  joint  and  coalition  forces,  capable  of  operating  at  high  tempos  and  able 
to  adapt  to  and  leverage  opportunities  as  they  emerge,  are  hallmarks  of  the  emerging 
future  force.  Many  of  these  characteristics  were  evident  in  Operations  Enduring  Freedom 
and  Iraqi  Freedom.  The  commercial  world  values  similar  characteristics.  The  ability  to 
sense,  process  and  make  mid-course  corrections  in  response  to  real-time  intelligence  is  a 
competitive  advantage  not  just  in  combat  but  also  in  business.  In  the  DOD,  we  talk  about 
“coherently  joint”;  in  the  commercial  world,  the  term  is  the  “extended  enterprise.” 

The  extended  enterprise  is  defined  as  “a  networked  supply  chain  that  integrates  partners, 
suppliers,  manufacturers,  retailers  and  customers  in  a  seamless,  Internet-based 
communications  system.”3  More  important,  it  entails  collaborative  behavior  among 
business  partners  and  thus  crosses  multiple  enterprises.  The  benefits  of  such 
collaborative  behavior  translate  directly  to  the  bottom  line  -  leaner  inventories,  lower 
working  capital,  higher  profits,  and  better  customer  service.4 

Implications  for  Systems  and  Programs 

How  do  these  trends  affect  the  systems  that  are  and  will  be  developed  to  meet  the  needs 
of  the  emerging  operating  environment,  be  it  in  government  or  commercial  sectors?  We 
see  several  significant  implications. 

First,  we  expect  to  see  a  continuing  trend  toward  increased  program  scale  and  scope  as 
single  acquisition  programs  encompass  what  in  the  past  would  have  been  separate 
acquisition  efforts.  Commercial  and  government  enterprises  are  also  seeking  to  integrate 
separate,  often  isolated,  operations,  processes  and  information.  In  so  doing,  they  take  an 
enterprise-wide  perspective  on  how  they  organize  and  operate.  Decisions  about 
investments  in  individual  information  technologies,  previously  made  locally,  are  now 
being  made  at  the  enterprise  level. 

A  related  trend  is  the  convergence  of  previously  separated  systems.  Programs  that  were 
previously  separately  managed  are  being  organized  into  cooperative  efforts.  For 
example,  the  Global  Command  and  Control  System  has  until  now  had  several  variants, 
each  focused  on  meeting  the  particular  needs  of  the  funding  organization.  The  current 
plan  is  to  converge  these  separate  efforts  (joint,  army,  maritime,  and  air  force)  into  a 
common  engineering  and  development  effort. 

The  combination  of  increased  scale  and  scope  and  convergence  of  previously  separated 
systems  translates  into  systems  that  will  cross  traditional  boundaries.  These  boundaries 
can  be  organizational,  functional,  or  disciplinary. 
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http://business.cisco.com/glossary 

4  Davis,  Edward,  and  Robert  Spekman,  Introduction  to  the  Extended  Enterprise:  Gaining  Competitive 
Advantage  through  Collaborative  Supply  Chains ,  chapter  published  on-line  at 
www.informit.com/isapi/product  id.,  Dec  12,  2003 
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Information  technologies  remain  at  the  core  of  these  emerging,  large-scale  systems,  as 
developers  seek  to  leverage  commercial  technologies  and  common,  often  commercial, 
standards.  To  that  extent,  there  will  be  a  continued  growth  in  integration  and  a 
commensurate  decline  in  custom  developments.  The  integration  challenge  will  continue 
to  increase  as  the  efforts  will  focus  on  integration  of  heterogeneous  components, 
separately  developed  and  managed.  Not  only  do  we  expect  the  components  to  be  diverse, 
but  the  development  activities  will  also  be  distributed  across  multiple,  often  physically 
dispersed,  activities  that  may  or  may  not  report  within  a  common  organizational 
structure. 

Further,  these  systems  will  need  to  accommodate  rapidly  evolving  needs,  organizational 
patterns,  and  technologies.  We  cannot  expect  to  be  able  to  articulate,  with  any  reasonable 
precision  and  certitude,  a  set  of  required  attributes  likely  to  remain  constant  over  the 
course  of  the  development  effort.  Rather,  we  fully  expect  that  the  needs  will  evolve  in 
parallel  with,  and  often  in  response  to,  the  evolution  of  the  systems  themselves. 

Finally,  these  systems  are  expected  to  be  increasingly  complex.  The  flip  side  of  having 
systems  that  accommodate  multiple  communities  and  interests  and  that  are  themselves 
evolving  is  that  the  system  behavior  will  not  always  be  predictable  but  instead  will 
emerge  as  a  result  of  the  interaction  of  the  components. 

The  Challenge  for  Systems  Engineering 

We  have  briefly  sketched  out  a  view  of  the  near  future  -  rapidly  evolving,  large-scale, 
massively  interconnected  systems  intended  to  bridge  traditional  boundaries.  These 
systems  are  not  just  scaled-up  versions  of  the  systems  that  we  have  been  developing  in 
the  latter  half  of  the  twentieth  century  but,  we  believe,  a  significant  departure.  The 
practice  of  systems  engineering  has  evolved  over  the  last  half  century  and  must  continue 
to  evolve  to  meet  the  challenges  of  this  new  class  of  systems.  We  have  concluded  that 
the  traditional  processes  and  practices  apply  only  in  part  to  this  new  class  of  “mega¬ 
systems.”  We  further  conclude  that  a  complementary  set  of  practices  and  processes  is 
needed. 

A  Framework  for  Exploring  Mega-systems 
A  Working  Definition 

“Mega-systems”  are  large,  potentially  complex  systems  that  are  formed  by  the  integration 
of  separately  developed  systems  to  provide  functionality  beyond  that  achievable  by  their 
component  systems.  Their  salient  characteristics  are  embedded  in  this  proposed 
definition. 


4 


First,  they  are  large,  man-made  systems.  While  “large”  is  clearly  a  relative  term,  we  mean 
something  generally  greater  in  scope  than  either  an  assembly5  or  even  a  large,  but  unitary, 
system  such  as  a  radar  or  even  an  entire  submarine. 

Second,  they  are  potentially  complex.  By  “complex,”  we  do  not  mean  that  they  are  either 
intricate  or  difficult  to  construct  (which  they  often  are),  but  rather  that  they  exhibit 
complex  behavior,  both  internally  among  their  components  and  as  a  whole. 

Third,  these  mega-systems  are  rarely  developed  as  a  monolithic  whole,  but  are  formed 
through  the  process  of  integration;  that  is,  they  are  “put  together.”  By  integration,  we 
mean  the  progressive  linking  and  testing  of  system  components  to  merge  their  functional 
and  technical  characteristics  into  a  comprehensive,  interoperable  system.  Note  that 
integration  includes  but  is  not  limited  to  the  ability  to  share  or  exchange  data. 

Finally,  these  systems  often  have  a  significant  human  dimension,  both  cognitive  and 
social,  which  contributes  both  to  the  complexity  of  behavior  and  to  the  continuing 
evolution  of  the  mega-system  as  it  operates  in  its  environment. 

The  Framework 

The  above  discussion  suggests  that  there  are  multiple  dimensions  to  understanding  mega¬ 
systems.  We  identify  four  key  dimensions. 

•  The  system  and  its  behavior,  ranging  from  linear  to  complex 

•  The  decision  making  environment,  ranging  from  unitary  to  pluralistic 

•  The  mission  environment,  ranging  from  stable  to  fluid  and  evolving 

•  The  acquisition  environment,  ranging  from  single  to  multiple  acquisitions 

The  framework,  shown  in  Figure  1,  highlights  the  first  three  of  these  dimensions. 


An  assembly  is  a  collection  of  components  and  modules  combined  into  a  single  unit.  A  typical  assembly 
may  perform  a  well-defined  function  within  a  larger  system,  hence  constituting  one  of  its  subsystems;  it  can 
also  be  an  independent,  self-contained  product  that  performs  a  single  function  of  a  limited  scale.  Examples 
of  assemblies  include  a  radar  receiver  or  a  computer  hard  disk.  See  Shenhar,  Aaron,  “A  New  Systems 
Engineering  Taxonomy,”  in  Proceedings  of  The  4th  Annual  International  Symposium  of  The  National 
Council  on  Systems  Engineering,  San  Jose,  CA,  August  1994. 
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Figure  1:  The  Basic  Framework6 


The  first  dimension,  System  and  Its  Behavior,  distinguishes  the  behavior  of  the  system  in 
terms  of  the  degree  of  complexity.7  Linear  systems  exhibit  behavior  that  is  regular,  well- 
understood  and,  to  a  large  extent,  predictable.  They  follow  well-established  rules  of 
behavior,  such  as  laws  of  physics  or  mechanics.  They  are  relatively  closed  to  the 
environment,  in  that  their  behavior  is  not  significantly  affected  by  events  external  to  the 
systems.  Finally,  their  component  elements  are  not  purposeful;  in  other  words,  they  exist 
only  as  part  of  the  larger  system  and  do  not  follow  their  own  independent  goals. 

In  contrast,  not  all  the  attributes  and  behavior  of  a  complex  system  are  directly 
observable  and  not  all  the  observable  interactions  are  understood.  Second,  they  do  not 
follow  well-ordered,  predictable  rules  of  behavior.  Solutions  to  specific  problems  may 
well  result  in  totally  unexpected  responses.  Third,  complex  systems  exhibit  emergent 
behavior,  in  that  the  interaction  of  components  results  in  behavior  that  can  not  be  only 
unexpected  but  sometimes  also  quite  different  from  the  behavior  of  the  components 
themselves.  Thus,  it  may  be  difficult  to  predict  the  effects  of  a  change  without  actually 
implementing  it.  Finally,  complex  systems  interact  with  their  environment  and  thus 
evolve  over  time.  Complex  systems  cannot  be  understood  merely  by  decomposing  them 
into  their  constituent  elements  and  separately  analyzing  these  elements.  Instead,  the  focus 
is  on  the  nature  and  effects  of  their  interactions  not  only  on  other  component  systems,  but 
also  on  the  whole. 

The  second  dimension,  Decision  Making  Environment,  addresses  the  extent  to  which 
decision  makers  agree  as  to  the  goals  and  objectives  of  the  system  as  a  whole.  Unitary 
decision  making  implies  agreement.  Decisions  are  made  and  implemented  in  accordance 
with  these  common  goals  and  are  thus  acceptable  to  all  stakeholders.  In  contrast,  decision 

6  This  framework  is  an  extension  of  ideas  presented  by  Michael  Jackon  and  P.  Keys,  “Towards  a  System  of 
Systems  Methodologies,  “  [sic],  J.  Opt.  Res.  Soc.,  Vol  35,  No.  6, 473-486,  1984. 

7  Systems  in  which  human  and  group  interactions  dominate  are  more  likely  to  exhibit  complex  behavior; 
systems  which  are  more  machine-like  are  more  likely  to  exhibit  linear  behavior. 
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making  is  pluralistic  if  there  is  little  or  no  agreement  as  to  the  goals  and  objectives  of  the 
mega-systems  and  decision  makers  instead  focus  on  their  local  concerns.  In  such 
instances,  the  few  decisions  made  will  address  only  those  aspects  on  which  the  various 
stakeholders  can,  in  fact,  reach  agreement.  On  occasion,  decisions  can  be  imposed  on  the 
stakeholders,  but  in  these  cases  either  blatant  or  more  subtle  pushback  could  be  expected. 

The  third  dimension  is  the  Mission  Environment.  It  can  range  from  one  that  is  stable  and 
enduring,  in  which  the  processes,  procedures,  and  relationships  are  well  understood  and 
likely  to  evolve  slowly,  to  one  that  is  fluid  and  dynamic,  where  participants,  their 
interactions,  and  the  “rules  of  the  game”  change  significantly  and  rapidly.  In  a  fluid, 
evolving  environment,  understanding  today’s  patterns  of  interaction  helps  little  in 
anticipating  future  patterns. 


As  you  move  from  the  lower  left  region,  where  system  behavior  is  linear,  the  decision 
making  environment  is  unitary,  and  the  mission  environment  is  stable,  the  behavior 
becomes  more  complex,  more  unpredictable  and  more  fragmented.  The  former  is  termed 
the  region  of  “well-bounded”  systems;  the  latter  is  the  region  of  “mega-systems.” 

Well-bounded  systems  lend  themselves  best  to  traditional  systems  engineering  and 
development  approaches,  whichCheckland  has  termed  these  approaches  “hard  systems 
thinking”.  Such  systems  are  based  on  “the  assumption  that  the  problem  task  they  tackle  is 
to  select  an  efficient  means  of  achieving  a  known  and  defined  end.”8  This,  in  turn,  is 
predicated  on  having  well-defined,  precise,  and  stable  requirements.  Because  of  the 
linear  nature  of  the  system’s  behavior,  traditional  systems  engineering  assumes  that 
overall  functions  can  be  decomposed  and  allocated  to  components  with  the  expectation 


g 

P.B.  Checkland,  “The  Origins  and  Nature  of  ‘Hard’  Systems  Thinking,” ,/.  Appl.  Systems  Analysis,  Vol.  5, 
1978,99-100. 
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that  the  components  and  the  whole  will  behave  as  expected.  The  engineer  can  predict 
and  therefore  has  some  measure  of  control  over  the  technical  interactions  of  the  system’s 
component  elements.  And  because  there  is  at  least  written  agreement  as  to  goals  and 
objectives,  the  manager  can  make  decisions  to  maximize  the  achievement  of  these 
desired  outcomes. 

In  contrast,  engineering  mega-systems  entails  dealing  with  top-level  requirements  that  are 
difficult  to  articulate  with  desired  precision,  are  often  internally  contradictory  and  will 
certainly  evolve  along  with  user  expectations.  The  engineering  process  also  must  deal 
with  functionality  and  behavior  that  will  emerge  from  the  interaction  of  the  components 
without  specific  direction  of  the  engineer.  Because  it  is  unpredictable,  such  behavior  is 
difficult  either  to  engineer  in  or  engineer  out.  Mega-systems  engineering  also  must  often 
deal  with  the  challenges  of  working  across  program  boundaries,  which  may  entail 
competition  for  limited  resources  and  between  alternative  solutions. 

Examples  of  Mega-Systems 

A  wide  range  of  activities  can  be  viewed  as  mega-systems,  encompassing  both  efforts 
that  set  out  to  compose  a  mega-system  from  elements  that  were  previously  developed  as 
stand-alone  systems  to  ones  that  set  out  to  design  a  fundamentally  new  capability,  in 
effect,  from  the  ground  up.  The  Air  and  Space  Operations  Center  and  law  enforcement 
information  sharing  are  examples  of  the  former  while  the  Army’s  Future  Combat 
Systems  and  the  Electronic  Product  Code  Network  are  examples  of  the  latter. 

The  Air  and  Space  Operations  Center  (AOC)  is  defined  as  the  Combined  Force  Air 
Component  Commander’s  “weapon  system”  for  commanding  air  and  space  forces.9  It 
consists  of  the  staff  (roughly  1000  to  2000  personnel);  the  processes  involved  in 
planning,  tasking  and  monitoring  all  air  operations  in  a  theater  of  operations;  and  the 
enabling  technologies.  These,  in  turn,  consist  of  over  80  different  elements  -  including 
infrastructure,  applications,  servers  and  databases  -  and  have  been  described  as  “different 
pieces  representing  different  fiefdoms  and  principalities.”10  These  elements  lack  a 
common  conceptual  basis,  common  funds  to  solve  cross-cutting  problems,  or  common 
control  or  management.  In  addition,  individual  elements  have  many  “customers”  of 
which  the  AOC  is  only  one,  and  have  evolved  independently  at  different  rates. 

Since  9/11,  there  has  been  a  clear  mandate  to  improve  information  sharing  among 
federal,  state,  and  local  law  enforcement  agencies.  The  scope  of  this  effort  potentially 
encompasses  not  only  multiple  federal  agencies  but  also  roughly  18,000  state  and  local 
law  enforcement  organizations.  Today,  these  organizations  use  hundreds  of  different 
systems,  both  home-grown  and  commercial.  But  perhaps  more  challenging  than  the 
technical  aspects  are  the  legal  and  cultural  barriers  to  information  exchange.  These 
barriers  include  not  only  growing  privacy  concerns  about  sharing  information  but  also 
mutual  distrust  among  different  organizations,  particularly  regarding  investigative  data. 


9  Rudolph,  Col  John,  “AN/USQ-163  Air  &  Space  Operations  Center  (AOC/’,  briefing. 

10  Norman,  Douglas  O.,  and  Michael  L.  Kuras,  “Engineering  Complex  Systems”,  chapter  in  Complex 
Engineered  Systems',  Purseus,  (in  press). 
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The  Future  Combat  Systems  (FCS)  is  the  Army’s  key  transformation  program.  It  is 
intended,  over  time,  to  replace  the  current  inventory  of  Abrams  tanks  and  Bradley 
Fighting  Vehicles  with  a  new  family  of  lighter  weight,  more  deployable  but  equally  as 
lethal  and  survivable  platforms.  It  is  described  as  “a  networked  “system  of  systems”  - 
one  large  system  made  up  of  18  individual  systems,  plus  the  network,  plus  the  soldier.”11 
These  individual  systems  encompass  not  only  manned  ground  vehicles  but  also 
unmanned  ground  and  air  platforms.  (Of  note,  some  in  the  Army  are  now  referring  to  the 
FCS  program  as  1+18  suggesting  that  the  network  that  will  link  the  various  platforms 
together  takes  precedence). 

The  Electronic  Product  Code  (EPC)  Network  is  an  open  standard,  global  network  using 
low-cost  radio  frequency  identification  (RFID)  tags  to  track  items  throughout  the  supply 
chain.  It  was  developed  over  a  period  of  four  years  by  a  consortium  of  universities, 
headed  by  the  Massachusetts  Institute  of  Technology,  and  was  cooperatively  funded  by  a 
sponsors  that  included  retailers,  consumer  packaged  goods  manufacturers,  and 
technology  vendors.  By  allowing  pallets,  cases,  and  even  individual  items  to  be  tracked 
globally,  this  technology  can  potentially  transform  the  supply  chain.  Adoption  of  the 
technology  has  been  greatly  accelerated  by  a  mandate  from  Wal-Mart,  requiring  its  top 
suppliers  to  begin  implementation  in  2005.  At  the  same  time,  vocal  concerns  from 
privacy  advocates  have  impacted  some  planned  field  trials. 

Preliminary  Observations 

One  of  the  challenges  of  examining  these  and  other  similar  mega-systems  is  that  they  are 
all  still  in  various  stages  of  development  and  therefore  it  is  difficult  to  consider  them  as 
formal  case  studies.  However,  several  observations  can  be  made. 

First,  no  single  engineering  technique  is  common  to  these  efforts.  For  example,  the  AOC 
has  attempted  and  subsequently  rejected  using  traditional  systems  engineering 
techniques.  FCS  is  following  the  Defense  Acquisition  University’s  “Systems  Engineering 
Fundamentals”  as  a  guide  for  the  program.  The  EPC  Network  efforts,  perhaps  because 
they  were  more  oriented  towards  technology  development,  did  not  explicitly  follow  any 
methodology  but  did  emphasize  early  prototyping  and  field  trials. 

Second,  there  is  no  common  organizational  framework.  There  is  a  Special  Project  Office 
in  charge  of  the  AOC  but,  in  effect,  there  are  many  component  systems  that  are  managed 
by  different  organizations  responding  to  different  constituencies.12  The  FCS  program  has 
hired  a  Lead  Systems  Integrator  who  has  total  systems  integration  responsibility  and  is 
responsible  for  managing  the  identification,  selection  and  procurement  of  major  systems 
and  subsystems.  By  contrast,  improved  law  enforcement  information  exchange  is 


1 1  http://www.boeing.com/defense-space/ic/fcs/bia/about.html 

12 

The  Air  Force  is  in  the  process  of  selecting  a  Lead  Service  Integrator.  This  LSI  is  not  expected  to 
provide  specifications  of  components  or  interfaces.  Instead,  the  LSI  is  expected  to  establish  and  oversee  an 
environment  in  which  components  are  gradually  but  continually  conceived,  implemented,  fielded  and 
evaluated.  See  http://herbb.hanscom.af.mil/esc  opps.asp?rfp=R495 
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accomplished  by  a  number  of  separately  managed  initiatives  while  the  EPC  Network 
effort  was  organized  as  a  collaborative  undertaking  that  was  funded  by  a  wide  range  of 
industry  sponsors,  a  number  of  whom  were  direct  competitors. 

Third,  organizational,  cultural,  political  and  other  “soft”  issues  have  had  critical,  often 
confounding  impacts  that  require  engineers  to  refine  their  visions  and  adjust  their  plans. 

Fourth,  while  it  is  possible  to  state  goals  as  broad  visions,  it  is  often  difficult  to  translate 
them  into  clear  and  unambiguous  statements  of  specific  desired  outcomes  and  to  hold 
those  outcomes  constant  in  the  face  of  changing  technologies,  expectations,  and 
constraints  or  even  new  opportunities. 

What  Seems  to  Work  Well...  and  Not  as  Well 

Techniques  that  seem  to  work  for  mega-systems  include: 

•  Engineering  enablers,  including  architectures,  visions,  and  plans  (as  long  as  they 
are  viewed  as  means  rather  than  the  end  itself) 

•  Techniques  that  facilitate  continuous,  broad-based  involvement  by  representatives 
from  multiple  organizations,  including  Integrated  Product  Teams  and 
collaboration  environments.  These  work  best  when  there  is  visible  support  from 
the  senior  leaders  of  the  represented  organizations. 

•  A  real  consensus  around  the  basic  infrastructure  and  the  key  design  tenets. 

•  Integration  facilities,  both  virtual  and  real,  that  allow  for  discovery  and 
exploration  of  unanticipated  behaviors. 

•  Early  field  trials  and  experiments  to  help  explore  how  the  elements  work  with  one 
another  and  to  introduce  “real  world”  dimensions. 

•  The  critical  role  of  a  charismatic  “champion”  who  is  able  to  forge  alliances  across 
organizational  boundaries  and  overcome  process  limitations. 

Techniques  that  do  not  seem  to  work  as  well: 

•  Efforts  to  develop  detailed  and  comprehensive  requirements  and  specifications 

•  Insufficient  attention  to  understanding  the  larger  environment  in  which  the  mega¬ 
system  will  operate  and  evolve;  not  involving  all  key  stakeholders. 

•  Multiple  stakeholders,  separate  agendas,  distrust 

•  Establishing  unnecessarily  complex  organizations;  emphasizing  process 

•  Developing  a  grand  design  and  expecting  it  to  remain  constant  in  the  face  of 
technology  obsolescence,  changing  user  expectations,  and  evolving  mission 
environments 

•  Technical  solutions  for  inherently  non-technical  problems  (e.g.,  privacy) 

13  Detailed  requirements  are  developed  to  scope  contractor  and  subcontractor  efforts  and  associated  costs. 
Alternative  contract  management  approaches  might  include  incentive  fees  which  reward  enablers  of  mega¬ 
system  development  as  well  as  management  reserves  for  course  corrections  identified  though  incremental 
experimentation.  This  is  a  substantial  topic  onto  itself  and,  while  it  is  outside  the  scope  of  this  paper, 
warrants  further  exploration. 
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Acquisitions  across  program  boundaries 


Implications  for  Engineering  Mega-Systems 

Traditional  systems  engineering  is  centered  on  developing  products  that  have  well- 
defined  boundaries  and  meet  pre-conceived  specifications.  Engineering  mega-systems  is 
considerably  messier.  It  must  deal  with  ambiguous  boundaries,  continuously  changing 
expectations  including  new  opportunities  that  were  not  initially  envisioned,  technology 
obsolescence  and  emergence,  and  a  shifting  mix  of  cooperation  and  competition  among 
participants  and  stakeholders.  However,  it  is  important  to  emphasize  that  mega-system 
engineering  does  not  replace  traditional  systems  engineering.  To  the  extent  that  mega¬ 
systems  encompass  well-bounded  elements,  traditional  systems  engineering  practices  will 
continue  concurrently  with  the  practices  that  will  evolve  around  mega-systems 
engineering. 

The  following  is  a  preliminary  set  of  implications  for  mega-systems  engineering.  These 
too  will  evolve  as  we  gain  experience  with  the  challenges  of  this  class  of  systems. 

Implications  #1.  Mega-systems  engineering  should  place  less  emphasis  on  having  a 
comprehensive,  detailed  set  of  requirements  and  specifications  at  the  onset  and  more 
emphasis  on  incremental  experimentation  and  trial.  Discovery  experiments,  using  early 
prototypes,  should  be  conducted  to  explore  the  boundary  between  linear  and  complex 
behavior  and  to  provide  early  insight  into  those  aspects  of  the  mega-system  that  are 
subject  to  emergent  behavior.  Field  trials  should  be  used  to  gain  insight  into  evolving 
user  expectations,  gather  lessons  learned  from  the  real  world,  and  better  understand  how 
the  capabilities  of  interest  fit  into  the  larger  context. 

Implication  #2.  Consensus  around  the  enabling  infrastructure  and  design  tenets  is  the 
structured  piece  of  the  unstructured  problem.  This  defines  the  minimum  set  of  standards 
and  architectural  guidelines  necessary  to  allow  different  elements  of  the  mega-system  to 
work  together  and  to  evolve  over  time.  It  enables  the  guided  evolution  of  the  mega¬ 
system  in  the  absence  of  a  comprehensive  grand  design. 

Implication  #3.  Mega-systems  engineering  should  make  maximum  use  of  existing 
collaborative  engineering  tools  and  practices  and  encourage  the  evolution  of  new 
techniques.  Collaborative  engineering  practices,  including  groupware  tools,  enable 
dispersed  teams  to  work  on  solutions  to  common  problems.  These  tools  and  techniques 
need  to  be  augmented  by  additional  tools  and  techniques  that  encourage  collaboration 
among  participants  who  view  themselves  are  competitors.  Institutional  and  contractual 
incentives  to  foster  collaboration  will  probably  be  required. 

Implication  #4.  Capabilities  that  are  deemed  useful  should  be  spiraled  off  to  the  user. 
Capabilities  that  can  be  provided  earlier  should  be  offered  to  the  user  in  a  meaningful 
“chunk.”  This  would  provide  not  only  a  measure  of  capability  that  was  not  previously 
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available,  but  also  provide  a  critical  mechanism  to  better  understand  user  preferences  and 
expectations. 

Implication  #5.  Mega-systems  should  be  encouraged  to  evolve  in  situ.  Users  will 
inevitably  adapt  the  capabilities  that  are  provided  to  them  to  meet  their  particular  needs 
and,  in  so  doing,  will  introduce  new  complexities,  both  in  how  they  use  the  technology 
provided  and  how  they  accomplish  their  missions.14 

Concluding  Thoughts  Intended  to  Provoke 

Network  centric  operations  and  extended  enterprises  entail  thinking  differently  about  the 
underlying  “business”  process.  By  analogy,  they  also  encourage  us  to  think  differently 
about  how  we  structure  and  implement  solutions.  Just  as  long  cycle  times  and  deliberate 
planning  are  hallmarks  of  industrial-age  business  strategies,  perhaps  long  acquisition 
cycles  and  grand  designs  are  the  hallmarks  of  industrial  age  product  developments.  And 
just  as  self-synchronizing  organizations  are  emerging  as  the  hallmark  of  the  information 
age,  perhaps  self-synchronizing  developments  based  on  agreed-to  goals  and  a  common 
infrastructure  -  and  not  on  detailed  specifications  -  will  become  the  hallmark  of  mega¬ 
systems  engineering. 


14  Woods,  David  D.,  “Steering  the  Reverberations  of  Technology  Change  on  Fields  of  Practice:  Laws  that 
Govern  Cognitive  Work”,  Plenary  Address,  Annual  Meeting  of  the  Cognitive  Science  Society,  August  10, 


2002 


12 


Systems  Engineering  in 
the  Information  Age:  The 
Challenge  of  Mega- 
Systems 

Renee  Stevens 
15  June  2004 


MITRE 


For  Internal  MITRE  Use 


©  2003  The  MITRE  Corporation.  All  rights  reserved 


Operation  Enduring  Freedom:  an  Early 
Glimpse  at  the  Future 

Unprecedented  Collaborative  Engagement  with  Networked  Forces 


■  SOF  forces  request  Close  Air  Support 


■  F-14  providing  Close  Air  Support  out  of 
weapons 

■  F-14  crew  employ  onboard  sensors  to 
mensurate  target 

■  Crew  passes  target  data  -  via  voice 
-  to  AWACS 


■  B-52  enabling  successful  target  kill  with 
precision  munitions 


■  Time  to  Target  18  Minutes 


•No  requirement  or  architecture  anticipated  it 


Not  achieved  by  any  single  system 


•May  never  happen  again  in  exactly  the  same  way 
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Demand  for  Agile,  Adaptive  Responses 


■  Convergence  of  multiple  trends 

Uncertainty  of  strategic  environment  demands 
agile/adaptive  response 

Seek  to  leverage  information  as  a  competitive 
source  of  power 

Information  revolution  provides  tools  to 
interconnect  a  wide  range  of  elements 

■  Computing/Storage 

■  Networking 

■  Wireless 

■  Assurance... 

Moving  from  bottom-up  requirements  to  top-down 
capability-based  acquisition  approach 

■  Leading  to  growing  emphasis  on  large-scale, 
richly  interconnected  “systems”  that  bridge 
traditional  organizational,  functional  and 
programmatic  boundaries...  " Mega-systems ” 

Same  logic  applies  in  defense,  government 
agencies  and  commercial  worlds 


Charles  Darwin  1809-1882 

“It  is  not  the  strongest  of  the 
species  that  survive,  nor  the 
most  intelligent,  but  the  ones 
most  adaptable  to  change” 
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The  Challenge  for  Systems  Engineering 


■  Trend  away  from  stand-alone  component  systems 

(platform-centric)  to  richly  interconnected  and  increasingly 
interdependent  systems  that  cross  traditional  boundaries 

-  System  of  Systems,  Family  of  Systems... 

-  Enterprise  and  “extended  enterprise”-wide  systems 


■  The  systems  we  are  being  called  on  to  help  acquire  appear 
to  be  qualitatively  distinguishable  from  those  that  have 
been  traditionally  and  successfully  addressed  using 
"traditional"  system  engineering. 

-  To  what  extent  do  the  SE  practices  and  processes  that 
evolved  in  post  WWII  era  apply  to  this  new  class  of  systems? 

-  Where  they  might  not  apply,  what  new  practices  and 
processes  might  be  required? 
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A  Working  Definition 


■  Mega-systems  defined  as  “large  scale,  potentially  complex 
systems  that  are  formed  by  the  integration  of  separately 
developed  systems  to  provide  functionality  beyond  that 
achievable  by  its  component  systems” 


■  Cross  traditional  boundaries  (organizational,  functional, 
programmatic...) 


■  Significant  human  dimension  which  contributes  to 

-  Complexity  of  behavior 

-  Continued  evolution 
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Framework  for  Exploring  Mega-Systems 


Little  or  no  agreement  as 
to  cormron  goals  and 
objectives;  decision 
makers  focus  on  local 
concerns 


Agreement  as  to  the  goals 
and  objectives;  decisions 
are  made  and  implemented 
WRT  common  goals 


Behavior  is  regular,  well 
understood  and,  to  a  large 
extent,  predictable 


Relatively  closed  to  the 
environment 


Linear  System  Behavior  Complex 


Information 
Systems  * 


Cognitive  Social 

Systemi  Systems 


Components  not  purposeful; 
exist  only  as  part  of  larger 
system 


Key  question:  How  to 
engineer  and  evolve  a 
mega-system  in  the 
absence  of  familiar 
control  mechanisms? 


Not  all  behavior  directly 
observable;  not  all 
interactions  are  well 
understood 

Do  not  necessarily  follow 
predictable  rules  of 
behavior;  solutions  to 
specific  problems  may  have 
totally  unexpected 
consequences 

I  nteract  with  environment 
and  evolve 
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Examples  of  Mega-Systems 

Air  Operations  Center  Federal,  State  &  Local  Info  Sharing 


1000-2000  people  working  around  the 
clock  in  2  shifts;  ~70K  ft2  of  space 

100+  workstations 

~$50M  of  planning  &  decision  aids  and 
infrastructure 

30-80  apps  being  continually  refreshed 
and  occasionally  upgraded  and 
augmented;  ~25  BIG  apps 


■  Federal  +  -1 8,000  state  &  local 
police  organizations,  majority  <10 
staff 

■  -680,000  law  enforcement  officers 

■  -100  different  systems  of  which 
>30  are  commercial 

■  Mutual  distrust  particularly  about 
sharing  case  information 


■  Generates  and  manages  24  hour 
schedules  for  the  application  of 
airborne  assets;  1  AOC  per  theatre. 


■  Some  successes  (fingerprint  file, 
national  criminal  information  file) 

■  Growing  privacy  concerns 
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Engineering  “Mega-systems”  is  Messy 


■  Ambiguous  boundaries 


■  Continuously  changing  expectations,  including  new 
opportunities  not  originally  envisioned 


■  Technology  obsolescence  and  emergence 


■  Shifting  mix  of  cooperation  and  competition  among 
participants  and  stakeholders 
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What  Seems  to  Work  Well...  And  Not  So  Wei 


■  The  enablers 

(Some)  architectures,  visions, 
engineering  master  plans... 

■  Continuous,  broad-based 
involvement 

Representatives  from  different 
organizations  actively  involved 

Visible  senior  leader  support 

■  Consensus  around  infrastructure 
and  tenets 

-  Open  standards 

■  Guided,  incremental 
developments 

■  Integration  facilities  (virtual  and 
real) 

■  Experimentation,  early  field  trials 

■  Response  to  real  crisis 

Overcome  “tribal”  tendencies 

■  Charismatic  “champion”  that  can 
overcome  process  limitations 


■  Requirements  and  specs 

Difficult  to  articulate  how  parts  will 
work  in  context  of  the  whole  -  lack 
lexicon 

Desire  for  global  specificity  and 
completeness 

■  Multiple  stakeholders,  overly 
complex  organizations 

Separate  agendas,  distrust... 

Process  takes  precedence 

■  Dealing  with  uncertainty 

■  Grand  design 

■  Too  long  a  horizon 

Technology  changes,  expectations 
change,  users  change... 

■  Too  narrow  a  view 

Ignoring  some  key  stakeholders 

Technical  solutions  for  non¬ 
technical  issues  (e.g.  privacy) 

■  Acquisition  across  boundaries 
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Implications  for  Engineering  Mega- 
Systems 


■  Implication  #1:  Less  emphasis  on  comprehensive,  detailed 
requirements  and  specifications  at  the  onset  and  more 
emphasis  on  incremental  experimentation  and  trial 

■  Implication  #2:  Consensus  around  the  enabling 
infrastructure  and  design  tenets  is  the  structured  piece  of 
the  unstructured  problem 

■  Implication  #3:  Make  maximum  use  of  existing  collaborative 
engineering  tools  and  practices  and  encourage  the 
evolution  of  new  techniques 

■  Implication  #4:  Capabilities  that  are  deemed  useful  should 
be  spiraled  off  the  use 

■  Implication  #5:  Encouraged  to  evolve  in  situ 
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Concluding  Thoughts  Intended  to  Provoke 

NCO  entails  thinking  differently  about  the  process;  we  should  also 
think  differently  about  how  we  structure  and  implement  solutions 


Military  Transformation* 

■What’s  In? 

Joint  and  Expeditionary 
mindset 

-  Ad  hoc  fight  in  flat 
organization 

■What’s  Out? 

Long  cycle  times 

Deliberate  planning 

Non-joint  Conops,  doctrine,  or 
operations 


Self-synchronizing  forces 
based  on  commander's  intent 


IT-Based  “Mega-Systems” 
■What’s  In? 

Complex  adaptive  systems 
mindset 

Capability  evolution  in  response 
to  unanticipated  events 

■What’s  Out? 

Long  acquisition  times 

-  Grand  design 

“Local”  requirements,  design,  or 
operations 

Self-synchronizing 
developments  based  on 
agreed-to  goals,  infrastmcture 
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